The Model-View-Controller (MVC) framework [1] allows the semantics and user-interface of application to be separated in different objects. As a result, the two components of the application can be created by developers with different skills. Moreover, the user-interface can be changed without changing the semantics. In multi-user interfaces, it allows (a) different users to see different views of the semantics and, (b) the code implementing the user interface of centralized semantic state to be placed on the computers of the local users. These properties have made it a fundamental software framework in interactive systems.
MVC was created without considering automation—the model, view and controller were expected to be implemented manually. Designers of recent commercial user-interface tools, such as Swing and Windows Forms, have shown that by making the tools directly aware of certain aspects of applications models such as their external structure, it is possible to increase automation. Awareness of model aspects is difficult when they are encapsulated. These tools have addressed this problem by being aware of only well-known models, that is, models whose types are defined by the tools. As a result, they provide limited abstraction flexibility. Various solutions to this problem have been proposed recently, but they compromise automation and user-interface flexibility. One contribution of the present subject matter is to show that this seeming contradiction between abstraction flexibility and the other two metrics—i.e., user automation and user interface flexibility, can be overcome by exploiting the following key observation: Just as there is uniformity in user-interfaces, there is also commonality in programming interfaces. In fact, the reason for both regularities is the same—to make it easy for users and programmers to understand and use these two kinds of interfaces. It has been found, somewhat surprisingly, that awareness of the conventions or patterns programmers follow to make their objects understandable to others can also be used to improve the tool in all three dimensions.
Motivating Pattern-Based UI Automation
Traditional user-interface tools have taken the approach of defining widgets such as text-fields and sliders without having any model of the higher-level application objects to which they are bound. Model-based systems [2, 3] take the converse approach of starting from semantic objects, and binding them to appropriate model-aware widgets—which are essentially views and controllers in the MVC terminology. These bindings can be programmer-defined or predefined. Several systems, including commercial ones such as Apple's Cocoa [4], Swing, and Windows Forms, provide a framework to register and lookup programmer-defined bindings. Automation of MVC, however, requires toolkit-supported bindings between a variety of toolkit-defined widgets and model objects.
Previous research has identified several useful bindings between widgets and semantic entities defined by traditional programming languages, such as the mapping between a menu (e.g. combo-box) and an enum [5], an editable customizable hierarchical form and a hierarchical record [6], and a button and a procedure [5]. Adapting this research to object-oriented languages, however, has proved to be difficult. The idea of model-aware widgets is fundamentally at odds with the notion of programmer-defined encapsulated objects. The former requires widgets to have some knowledge of the model such as its logical components, while the latter hides all details of the model from external objects except the set of methods that can be invoked on it.
One way to address this issue is to support only toolkit-defined models. This approach is supported by several commercial user-interface tools such as Swing, which supports tree, table, and combo-box widgets for models implementing programming interfaces defined by these widgets. The model-aware widgets augment rather than replace model-aware widgets, thereby increasing automation without reducing user-interface flexibility. Commercial toolkits have focused only on model substitutability—allowing the same widget to be used to interact with instances of different model classes that implement the same interface. They do not currently support widget substitutability—allowing the same model to be displayed using different model-aware widgets. Recent work [7] has shown that is possible for a toolkit to support both model and widget substitutability by defining a single model interface for a variety of structured widgets including trees, tables, tabbed composition of widgets, forms, and browsers. However, like the commercial tools, it supports encapsulated but not programmer-defined types. The dual of this idea, implemented in [8], is to support programmer-defined but not encapsulated model types, that is, require programmers to create public instance variables in the model classes they define. The two dual ideas essentially require that programmers do conventional programming using an object-oriented language.
One popular approach to address this problem is to require programmers to provide a special description of the encapsulated models they create [9, 10] in some model-description language, and manually map references to this description to the implementation of the model in the object-oriented language. For example, in [9, 11], the programmers are responsible for manually mapping between the implementation of an object in a standard programming language and a manually specified XML-based description of it understood by the tool. The cost of carrying out these tasks when a model is first created, and each time it is changed, can be significant, especially for complex model types. It is perhaps for this reason that these tools, to the best of our knowledge, support only one-level model structures, that is, restrict model components to be primitive types. As a result, they cannot be used, for example, to define the model of an arbitrarily nested tree-widget such as a folder.
As described above, each of the above approaches makes a significant compromise. The reason for making the compromise is not surprising—the UI automation offered by model-aware widgets can result in considerable productivity gains given the high cost of manually implementing user interfaces using model-unaware widgets [12], which involves translating between potentially complex model and widget structures. These compromises result from the fundamental tension between model-awareness and encapsulation, which prevents toolkits from understanding arbitrary objects.
As described below, programming patterns have been used earlier in other contexts. One contribution of the present subject matter is the application of programming partners in implementation of user-interfaces of encapsulated models. In the next section, previous work to motivate the general idea of using programming patterns in a user-interface tool is surveyed. Next, the idea is developed by discussing a variety of new ways in which programming patterns can be used, along with associated architectures. The discussion of automatic generation of user interfaces by extending the MVC architecture ends by outlining a Java-based implementation of these concepts and experiences with it, and giving conclusions and directions for future work.
Accordingly, there exists a need for methods, systems, and computer readable media for automatically generating user interfaces using programming patterns.